Software applications often require concurrent execution of work items in multiple operating system threads. For the purpose of this application, work items are a decomposition of application functions into individual units of work. An example of a work item is counting the words in a file. Work items can be composed together to form a workflow. Workflows can then be re-used as work items inside other workflows. An example of such a reuse is counting all the words in all of the files in a directory.
There are physical limitations governing the maximum number of operating system threads that can be used by a process running a software application. These physical limitations are well documented for each operating system type and are outside the scope of this application. Software applications typically manage such limitations by organizing threads into pools served by queues. Threads in a pool typically “take” work items from the queue and execute them.
When work items are decomposed into other work items that also require concurrent execution, using one single thread pool for all the concurrency needs makes it hard to ensure that certain workflows do not exceed a predefined level of concurrency. In addition, there is a potential for deadlock and non-uniform patterns of execution. For example, if the execution of a work item needs to wait for the completion of other work items, deadlock can occur in the situation where all the threads in a thread pool end up waiting for other work items.